Violator Spaces: Structure and Algorithms 



B. Gartner a , J. Matousek b , L. Riist a , P. Skovron b 

a Institute of Theoretical Computer Science, ETH Zurich, 8092 Zurich, 

Switzerland 

b Department of Applied Mathematics and Institute of Theoretical Computer 
Science, Charles University, Malostranske nam. 25, 118 00 Praha 1, Czech 

Republic 



Abstract 

Sharir and Welzl introduced an abstract framework for optimization problems, 
called LP-type problems or also generalized linear programming problems, which 
proved useful in algorithm design. We define a new, and as we believe, simpler 
and more natural framework: violator spaces, which constitute a proper general- 
ization of LP-type problems. We show that Clarkson's randomized algorithms for 
low-dimensional linear programming work in the context of violator spaces. For 
example, in this way we obtain the fastest known algorithm for the P -matrix gen- 
eralized linear complementarity problem with a constant number of blocks. We also 
give two new characterizations of LP-type problems: they are equivalent to acyclic 
violator spaces, as well as to concrete LP-type problems (informally, the constraints 
in a concrete LP-type problem are subsets of a linearly ordered ground set, and the 
value of a set of constraints is the minimum of its intersection). 
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1 Introduction 



The framework of LP-type problems, invented by Sharir and Welzl in 1992 [1], 
has become a well-established tool in the field of geometric optimization. Its 
origins are in linear programming: Sharir and Welzl developed a randomized 
variant of the dual simplex algorithm for linear programming and showed that 
this algorithm actually works for a more general class of problems they called 
LP-type problems. 

For the theory of linear programming, this algorithm constituted an important 
progress, since it was later shown to be subexponential in the RAM model [2]. 
Together with a similar result independently obtained by Kalai [3], this was 
the first linear programming algorithm provably requiring a number of arith- 
metic operations subexponential in the dimension and number of constraints 
(independent of the precision of the input numbers). 

For many other geometric optimization problems in fixed dimension, the al- 
gorithm by Sharir and Welzl was the first to achieve expected linear runtime, 
simply because these problems could be formulated as LP-type problems. The 
class of LP-type problems for example includes the problem of computing the 
minimum-volume ball or ellipsoid enclosing a given point set in R d , and the 
problem of finding the distance of two convex polytopes in R d . Many other 
problems have been identified as LP-type problems over the years [2,4,5,6,7]. 

Once it is shown that a particular optimization problem is an LP-type prob- 
lem, and certain algorithmic primitives are implemented for it, several efficient 
algorithms are immediately at our disposal: the Sharir-Welzl algorithm, two 
other randomized optimization algorithms due to Clarkson [8] (see [9,10] for a 
discussion of how it fits the LP-type framework), a deterministic version of it 
[10], an algorithm for computing the minimum solution that violates at most 
k of the given n constraints [11], and probably more are to come in the future. 

The framework of LP-type problems is not only a prototype for concrete opti- 
mization problems, it also serves as a mathematical tool by itself, in algorith- 
mic [12,13] and non-algorithmic contexts [14]. 

An (abstract) LP-type problem is given by a finite set H of constraints and 
a value w(G) for every subset G C H . The values can be real numbers or, for 
technical convenience, elements of any other linearly ordered set. Intuitively, 
w(G) is the minimum value of a solution that satisfies all constraints in G. 
The assignment G h- > w(G) has to obey the axioms in the following definition. 

Definition 1 An abstract LP-type problem is a quadruple (H, w, W, <), where 
H is a finite set, W is a set linearly ordered by <, and w:2 H W is a map- 
ping satisfying the following two conditions: 
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Monotonicity: for all F C G C H we have w(F) < w(G), and 
Locality: for all F C G C and all h & H with w(F) = w(G) and 

w(G) < w(G U we have w(F) < w(F U {h}). 

As our running example, we will use the smallest enclosing ball problem, 
where if is a finite point set in R d and w(G) is the radius of the smallest 
ball that encloses all points of G. In this case monotonicity is obvious, while 
verifying locality requires the nontrivial but well known geometric result that 
the smallest enclosing ball is unique for every set. 

It seems that the order < of subsets is crucial; after all, LP-type problems 
model optimization problems, and indeed, the subexponential algorithm for 
linear programming and other LP-type problems [2] heavily relies on such an 
order. 

A somewhat deeper look reveals that often, we only care whether two subsets 
have the same value, but not how they compare under the order <. The 
following definition is taken from [1]: 

Definition 2 Consider an abstract LP-type problem (H,w,W,<). We say 
that B C H is a basis if for all proper subsets F C B we have w(F) ^ w(B). 
For G C H, a basis of G is a minimal subset B of G with w(B) = w(G). 

We observe that a minimal subset B C G with w(B) = w(G) is indeed a basis. 

Solving an abstract LP-type problem (H,w,W,<) means to find a basis of 
H. In the smallest enclosing ball problem, a basis of H is a minimal set B of 
points such that the smallest enclosing ball of B has the same radius (and is 
in fact the same) as the smallest enclosing ball of H, w(B) = w(H). 

In defining bases, and in saying what it means to solve an LP-type problem, 
we therefore do not need the order <. The main contribution of this paper is 
that many of the things one can prove about LP-type problems do not require 
a concept of order. 

We formalize this by defining the new framework of violator spaces. Intuitively, 
a violator space is an LP-type problem without order. This generalization 
of LP-type problems is proper, and we can exactly characterize the violator 
spaces that "are" LP-type problems. In doing so, we also establish yet another 
equivalent characterization of LP-type problems that is closer to the applica- 
tions than the abstract formulation of Definition 1. In a concrete LP-type 
problem, the constraints are not just elements of a set, but they are associ- 
ated with subsets of some linearly ordered ground set X, with the minimal 
elements in the intersections of such subsets corresponding to "solutions" . The 
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framework of concrete LP-type problems is similar to the model presented in 
[5] as a mathematical programming problem, with a few technical differences. 

These are our main findings on the structural side. Probably the most sur- 
prising insight on the algorithmic side is that Clarkson's algorithms [8] work 
for violator spaces of fixed dimension, leading to an expected linear-time algo- 
rithm for "solving" the violator space. Clarkson's algorithms were originally 
developed for linear programs with small dimension. They can be generalized 
for LP-type problems [9,10]. The fact that the scheme also works for violator 
spaces may come as a surprise since the structure of violator spaces is not 
acyclic in general (in contrast to LP-type problems). The LP-type algorithm 
from [2] is also applicable to violator spaces, but its analysis breaks down. 

We give an application of Clarkson's algorithms in the more general setting by 
linking our new violator space framework to well-known abstract and concrete 
frameworks in combinatorial optimization. For this, we show that any unique 
sink orientation (USO) of the cube [15,16,17,18,19,20,21,22,23,24] or the more 
general grid [16] gives rise to a violator space, but not to an LP-type problem in 
general. Grid USO capture some important problems like linear programming 
over products of simplices, generalized linear complementarity problems over 
P- matrices [16] or games like parity, mean-payoff, and simple stochastic games 
[25,26,27]. 

We show that we can find the sink in a unique sink orientation by solving the 
violator space, for example with Clarkson's algorithms. A concrete new result 
is obtained by applying this to P-matrix generalized linear complementarity 
problems. These problems are not known to be polynomial-time solvable, but 
NP-hardness would imply NP=co-NP [28,16]. Since any P-matrix generalized 
linear complementarity problem gives rise to a unique sink orientation [16], 
we may use violator spaces and Clarkson's algorithms to solve the problem in 
expected linear time in the (polynomially solvable) case of a fixed number of 
blocks. This is optimal and beats all previous algorithms. 

The rest of the paper is organized as follows. In Section 2, we formally define 
the frameworks of concrete LP-type problems and violator spaces, along with 
their essential terminology. Then we state our main structural result. 

In Section 3, we prove this result by deriving the equivalence of abstract and 
concrete LP-type problems, and of acyclic violator spaces. 

Section 4 shows that Clarkson's algorithms work for (possibly cyclic) violator 
spaces. Section 5, finally, shows how unique sink orientations induce violator 
spaces. A unique sink orientation can be cyclic, and a cyclic orientation gives 
rise to a cyclic violator space. Unique sink orientations are therefore nontrivial 
examples of possibly cyclic violator spaces. 
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2 Structural Results 



2.1 Concrete LP-type problems. 

Although intuitively one thinks about w(G) as the value of an optimal solution 
of an optimization problem, the solution itself is not explicitly represented 
in Definition 1. In specific geometric examples, the constraints can usually 
be interpreted as a subset of some ground set X of points, and the optimal 
solution for G is the point with the smallest value in the intersection of all 
constraints in G. For example, in linear programming, the constraints are 
halfspaces, the value is given by the objective function, and the optimum is 
the point with minimum value in the admissible region, i.e., the intersection of 
the halfspaces. In order to have a unique optimum for every set of constraints 
(which is needed for w to define an LP-type problem), one assumes that the 
points are linearly ordered by the value; for linear programming, we can always 
take the lexicographically smallest optimal solution, for instance. 

Such an interpretation is possible for the smallest enclosing ball problem too, 
although it looks a bit artificial. Namely, the "points" of X are all balls in H d , 
where the ordering can be an arbitrary linear extension of the partial ordering 
of balls by radius. The "constraint" for a point h e H is the set of all balls 
containing h. 

The following definition captures this approach to LP-type problems. 

Definition 3 A concrete LP-type problem is a triple (X, -<,'H), where X is 
a set linearly ordered by <, H. is a finite multiset whose elements are subsets 
of X , and for any Q C 7Y 7 if the intersection f] Q := Cloeg G ^ s nonempty, then 
it has a minimum element with respect to ^ (for Q = we define f]Q :— X). 

The definition allows H to be a multiset, i.e., a constraint set A C X may 
be included several times. For example, in an instance of linear programming, 
some constraints can be the same, which we can reflect by this. In Subsection 
3.4 we provide an example of an abstract LP-type problem, for which the 
multiplicity comes in handy to represent it as a concrete LP-type problem. 

A similar model has been presented in [5] (mathematical programming prob- 
lem). The slight difference is that it allows several points to have the same 
value but the constraints form a set rather than a multiset. 

Bases are defined analogously to Definition 2. 

Definition 4 Consider a concrete LP-type problem (X,^.,H). We say that 
B C Ti is a basis if for all proper submultisets T C B we have min(f| T) -< 
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min(n B) . For Q C Ti, a basis of Q is a minimal B C Q with min(n B) = 
min(n^). 

As before, a minimal B C Q with min(f|i3) = min(f|^) is indeed a basis. 

Given any concrete LP- type problem V = (X, ^,Ti), we obtain an abstract 
LP-type problem P = (Ti, w, X, ;<) according to Definition 1 by putting 
w{Q) = min(fl^) (or w{Q) = +00, if f]Q is empty), as is easy to check 
(proof omitted). It is clear that B C Q is a basis of Q in P if and only if B is 
a basis of Q in P. We say that P is basis- equivalent to V . 

Somewhat surprising is the converse, which we prove below in Theorem 8: Any 
abstract LP-type problem (H, w, W, <) has a "concrete representation", that 
is, a concrete LP-type problem that is basis-equivalent to (H,w, W, <). 

Strictly speaking, if the multiset Ti in the concrete LP-type problem has el- 
ements with multiplicity bigger than 1, then Ti cannot be used as the set of 
constraints for the abstract LP-type problem (since it is not a set). However, 
we can bijectively map Ti to a set, i.e., we take any set H with \H\ = \7i\ 
and a mapping /: H — > TL such that for any h G Ti, the number of elements 
h G H that map to h is equal to the multiplicity of h. For G C H we then 
define w{G) = mm(f] geG f(g)) which gives us a fair abstract LP-type problem 
P = (H,w,X, ^) basis-equivalent to V. In this case, by basis-equivalence we 
mean the existence of a suitable mapping / together with the condition that 
B C G is a basis of G in P if and only if the multiset {fib): b G B} is a basis 
of {/(«?): geG} in P. 

2.2 Violator spaces. 

Let (i7, w, W, <) be an abstract LP-type problem. It is natural to define that 
a constraint h G H violates a set G C if of constraints if u>(G U {/i}) > w(G). 
For example, in the smallest enclosing ball problem, a point /i violates a set 
G if it lies outside of the smallest ball enclosing G (which is unique). 

Definition 5 The violator mapping of (H,w,W,<) is defined by V(G) = 
{h G H: w(GU{h}) > w(G)}. Thus, V(G) is the set of all constraints violating 
G. 

It turns out that the knowledge of V(G) for all G C H is enough to describe the 
"structure" of an LP-type problem. That is, while we cannot reconstruct W, <, 
and w from this knowledge, it is natural to consider two LP-type problems with 
the same mapping V: 2 H — > 2 H the same (isomorphic). Indeed, the algorithmic 
primitives needed for implementing the Sharir-Welzl algorithm and the other 
algorithms for LP-type problems mentioned above can be phrased in terms of 



6 



testing violation (does h E V(G) hold for a certain set G C HI), and they 
never deal explicitly with the values of w. 

We now introduce the notion of violator space: 

Definition 6 A violator space is a pair (H, V), where H is a finite set and V 
is a mapping 2 H — > 2 H such that 

Consistency: G fl V(G) = holds for all G C H , and 
Locality: for all F C G C H , where G fl V(F) = ; we have 

V(G)=V(F). 

A basis of a violator space is defined in analogy to a basis of an LP-type 
problem. 

Definition 7 Consider a violator space (H, V) . We say that B C H is a basis 
if for all proper subsets F C B we have B fl V(F) ^ 0. For G C H, a basis of 
G is a minimal subset B of G with V(B) = V(G). 

Observe that a minimal subset B C G with V(5) = V(G) is indeed a basis: 
Assume for contradiction that there is a set F C -B such that B fl V(F) = 0. 
Locality then yields V(£?) = V(F) = V(G), which contradicts minimality of 
B. 

We will check in Subsection 3.2 that the violator mapping of an abstract LP- 
type problem satisfies the two axioms above. Consistency is immediate: since 
w(G) = w(G U {h}) for ft 6 G, no element in G violates G. The locality 
condition has the following intuitive interpretation: adding only non-violators 
to a set does not change the value. 

We actually show more: given an abstract LP-type problem (H,w, W, <), the 
pair (H, V), with V being the violator mapping, is an acyclic violator space. 
(Acyclicity of a violator space will be defined later in Definition 10.) It turns 
out in Subsection 3.3 that acyclicity already characterizes the violator spaces 
obtained from LP-type problems, and thus any acyclic violator space can be 
represented as an LP-type problem (abstract or concrete). These equivalences 
are stated in our main theorem. 

Theorem 8 The axioms of abstract LP-type problems, of concrete LP-type 
problems, and of acyclic violator spaces are equivalent. More precisely, every 
problem in one of the three classes has a basis- equivalent problem in each of 
the other two classes. 

The construction is illustrated on simple instances of problems of linear pro- 
gramming and the smallest enclosing ball in Subsection 3.4. Several more 
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results concerning violator spaces have been achieved in the MSc. thesis of the 
fourth author [29]. 



3 Equivalence of LP-type Problems and Acyclic Violator Spaces 

In this section we prove Theorem 8. 
3. 1 Preliminaries on Violator Spaces 

To show that every acyclic violator space (H, V) originates from some concrete 
LP-type problem, we need an appropriate linearly ordered set X of "points" , 
and then we will identify the elements of H with certain subsets of X. 

What set X will we take? Recall that for smallest enclosing balls, X is the set 
of all balls, and the subset for h G H is the subset of balls containing h. It is 
not hard to see that we may restrict X to smallest enclosing balls of bases; 
in fact, we may choose X as the set of bases, in which case the subset for h 
becomes the set of bases not violated by h. 

This also works for general acyclic violator spaces, with bases suitably ordered. 
The only blemish is that we may get several minimal bases for G C if; for 
smallest enclosing balls, this corresponds to the situation in which several 
bases define the same smallest enclosing ball. To address this, we will declare 
such bases as equivalent and choose X as the set of all equivalence classes 
instead. 

In the following, we fix a violator space (H, V) . The set of all bases in (H, V) 
will be denoted by B. 

Definition 9 B,C G B are equivalent, B ~ C, ifV(B) = V(C). 

Clearly, the relation ~ defined on B is an equivalence relation. The equivalence 
class containing a basis B will be denoted by [B]. 

Now we are going to define an ordering of the bases, and we derive from this 
an ordering of the equivalence classes as well as the notion of acyclicity in 
violator spaces. 

Definition 10 For F,G C H in a violator space (H, V), we say that F < G 
(F is locally smaller than G) if F n V(G) = 0. 

For equivalence classes [B], [C] G £>/ ~ ; we say that [B] < [C] if there exist 
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Fig. 1. A cyclic violator space. 
B' E [B] and C E [C] such that B' < C . 

We define the relation <i on the equivalence classes as the transitive closure of 
<o- The relation <i is clearly reflexive and transitive. If it is antisymmetric, 
we say that the violator space is acyclic, and we define the relation < as an 
arbitrary linear extension of <\. 

The intuition of the locally- smaller notion comes from LP-type problems: if no 
element of F violates G, then GU F has the same value as G (this is formally 
proved in Lemma 11 below), and monotonicity yields that value- wise, F is 
smaller than or equal to G. 

Note that in the definition of [B] <o [C] we do not require B' <o C' to hold 
for every B' and C' . In fact B' C ma y happen for some bases B' and C', 
but C' < B' can not hold (which can easily be shown). 

To show that acyclicity does not always hold, we conclude this section with 
an example of a cyclic violator space. 

We begin with an intuitive geometric description; see Figure 1. We consider a 
triangle without the center point. We say that a point is "locally smaller" if it 
is farther clockwise with respect to the center. The constraints in our violator 
space are the three halfplanes /, g, h. 

The locally smallest point within each halfplane is marked, and a halfplane 
violates a set of halfplanes if it does not contain the locally smallest point in 
their intersection. 

Now we specify the corresponding violator space formally. We have H = 
{/, g, h}, and V is given by the following table: 
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This (H, V) is really a violator space, since we can easily check both consistency 
and locality. The bases are 0, one-element sets, and H. We have {/} <o {h} <q 
{g} <o {/}; but none of the one-element bases are equivalent; i.e., <i is not 
antisymmetric. 

3.2 Abstract LP-type Problems yield Acyclic Violator Spaces 

In this subsection, we show that the violator mapping of an abstract LP-type 
problem is an acyclic violator space. To this end, we need the following two 
lemmas. 

Lemma 11 Consider an abstract LP-type problem (H,w, W, <) with violator 
mapping V. Let A, B C H, where B is not violated by any h G A (AC\V(B) = 
0;. Then w(AU B) = w{B). 

PROOF. From monotonicity, we immediately obtain the inequality ">" . The 
inequality "<" can be shown by induction on \A\. If \A\ = 1, i.e., A = {h}, then 
w(BU{h}) > w(B) would imply that B is violated by h G A, a contradiction. 

Let \A\ > 1 and A = A 0{h} (disjoint union). From the induction hypothesis 
we have w(B U A ) = w(B). Now, if w(B U A ) < w(B U A U {h}), then 
by locality (for B U Aq, B and h) we get w(B) < w(B U {h}). This means 
that h G V{B), and since h G A we have h G A n V(5), a contradiction. 
So w(B) = w(B U Aq) > w(B U A U {h}) = w(B U A). We have proved 
w{AUB)<w{B). □ 

Lemma 12 Consider an abstract LP-type problem (H,w, W, <) with violator 
mapping V. Then for any A,BCH with V(A) = V(5) we have w(A) = w(B) . 
Conversely, w(A) = w(B) = w(A\JB) implies V(A) = V(_B). In particular, if 
AC B and w{A) = w(B), then V(A) = V(B). 

Note that the condition w(A) = w(B) generally does not suffice for V(A) = 
V(B). For example, having any H, we can define w by w(G) = \G\ for all 
G C H (it can be checked that it is an abstract LP-type problem). Then 
any G's of the same size have the same w, however, V(G) = H\G, and so no 
distinct G"s share the value of V. Roughly speaking, the equality w(A) = w(B) 
may hold just "by accident". This is one way in which we can see that w by 
itself does not reflect the combinatorial structure of the problem in a natural 
way. 

PROOF of Lemma 12. Let w(A) ^ w(B). Without loss of generality we 
assume w(A) > w(B) (note that here we use the linearity of the ordering 
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<). If A fl y(B) = 0, from Lemma 11 we would get w(A U B) = w(B), which 
contradicts w(AUB) > w(A) > w(B). So there necessarily exists h E AnV(B), 
but since h E A, we have h £ V(A). So V(A) ^ V(B). 

Conversely, suppose w(A) = w(B) = w(A U B). We want to show \I(A) = 
V(£), i.e., that w(A) < w{A U {h}) holds iff w(5) < w(B U {/i}) holds. 
By symmetry, it suffices to show only one of the implications. We assume 
w(A) < w(Au{h}). Then w(AUB) = w(A) < w(AU{h}) < w(AUBU{h}). 
Since B C A U B and w(B) = w(A U -B), we may use locality, which gives 
w(B) < w(B U {h}). So the desired equivalence holds. □ 

Proposition 13 Consider an abstract LP-type problem (H,w, W, <), and let 

V be its violator mapping. Then (H, V) is an acyclic violator space. Moreover, 
(H, V) is basis- equivalent to (H, w, W, <). 



PROOF. Clearly G fl V(G) = 0, since w(G U {g}) = w(G) for any g e G, 
so consistency holds. If G fl V(F) = for F C G, then by Lemma 11 we get 
w(F UG) = iu(F). Since F C G, we have FUG = G and so w{G) = w(F). 
Lemma 12 then yields V(G) = V(F), so locality holds. 

We proceed to prove acyclicity of (H, V). Fix [B] and [C], [B] ^ [C], with 
[£] < [C], that is 5' n V(C") = for some B' E [B] and C' E [C\. Lemma 11 
implies w(C') = w(B' U C'). For contradiction, assume w(B') > w(C); then 
iu(S') > w(B' U C") which with monotonicity yields w(B') = w(B' U C') = 
w(C'). Lemma 12 gives V(fi / ) = V(C"), a contradiction to [£] ^ [C]. Thus 
[S] < [C] for [B] ^ [C] implies w(B') < w{C') for some bases B' and C" 
out of the respective equivalence classes. By Lemma 12, w(B') is the same for 
all B' E [B] (because all bases in [B] have the same violators). Therefore, by 
chaining several < 's we also get w(B') < w(C) for [B] <i [C\. This proves 
that <i is necessarily antisymmetric (since < is an ordering of W). 

Finally, observe that by Lemma 12, B C G is an inclusion-minimal subset of 
G with w(B) = w(G) if and only if B is an inclusion-minimal subset of G 
with V(S) = V(G). So, B C G is a basis of G in (//, w, W, <) if and only if B 
is a basis of G in (H,V). Thus (H,V) is basis-equivalent to (H,w, W, <). □ 

At first glance, one might think that for F C G we should have V(F) D V(G). 
Unfortunately, this is not the the linear programming example in 

Figure 2 shows (the y-coordinate is to be minimized). 

We put F = {hi, /12} and G = {h±, hi, ^3} ^ F. The point 1 is minimum in 
the intersection of F , and 2 is minimum in the intersection of G. We have 
1 E h*, 2 £ /i*, and so /i* ^ V(F) and /i* G V(G). 
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ha 



Fig. 2. A linear programming example (F = {/ii,/i2} Q G = {h±, h,2, h^} with 
V(G)P(F)). 

3.3 Acyclic Violator Spaces yield Concrete LP-type Problems 

The following proposition is the last ingredient for Theorem 8. 

Proposition 14 Every acyclic violator space (H, V) can be represented as a 
concrete LP-type problem that is basis- equivalent to (if, V). 

PROOF. We are given an acyclic violator space (H, V) and we define the 
mapping S: H -> 2 e /~ that will act as a "concretization" of the constraints in 
H: 

S(h) = {[B]:B e B, h^V(B)}. 

Further, let Ti be the image of the mapping S taken as a multiset, i.e., 
Ti = {S(h):he H}. 

Thus, S is a bijection between H and Ti. By saying that a mapping S is 
a bijection between a set and a multiset we mean that for any h G Ti., the 
number of h G H that map to h is equal to the multiplicity of h. Note that 
we cannot use some common properties of set bijections; for instance we have 
to avoid using the inverse mapping S* -1 . 

Additionally, let a be the induced bijection of 2 H and 2 n defined by o~(G) = 
{S(h):h G G}, for G C H. 

Now, consider the triple <, Ti), where < is an arbitrary linear extension 

of <i (such an extension exists since (H,V) is acyclic and <i therefore anti- 
symmetric). This is a concrete LP-type problem: The only thing to check is the 
existence of a minimal element of every nonempty intersection f]Q {Q C Ti), 
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which is guaranteed by the linearity of < (remember from Definition 3 that 

n£:=n Gee G). 



It remains to prove basis-equivalence, which we do with the following two 
lemmas. 

Lemma 15 // B is an inclusion-minimal subset of G with V(S) = V(G) 
in (H,V) (that is, B is a basis of G), then min(r|o"(-B)) = min(n cr(G)) in 

(B/~,<,W). 

PROOF. It is clear that [B] G f]o-(G). Therefore, showing that there is 
no other basis in f]a(G) that is locally smaller than [B] proves the lemma, 
because then mm(f\a(G)) = [B] = min(f|o-(-B)) (the second equality holds 
since B is a basis of B; just replace G by B in the following proof). Assume 
for contradiction that a C with [C] ^ [B], [C] G C\v(G) and C < [B] exists. 
By [C] G fl cr(G) we have G fl V(C) = 0, which is equivalent to 

(Guc)n v(C) = 0, 

and by C < [B] we have C fl V(£>) = which is equivalent to (because B is 
a basis of G) 

(GuC)nv(B) = 0. 

Applying locality in (H, V) to these two equations tells us that V(C) = V(£?), 
a contradiction to [C] ^ [B\. □ 

Lemma 16 Ifa(B) is an inclusion-minimal submultiset of a(G) withmm.{p\a(B)) = 
min(r|cr(G0) in (£>/~,<,7Y) (that is, cr(B) is a basis of a{G)), then V(_B) = 
V(G)in(H,y). 

PROOF. Let A be a basis of B, so V(A) = V(B). Note that [A] G C\<r{B). 
Let [C] = jmn(C\cr(B)), thus B n V(C) = and therefore also A n V(C) = 0. 
This means that [A] < [C] from which we conclude that [A] = [C\. From 
min(ncr(G)) = [C] we get G fl V(C) = which is equivalent to 

GnV(B) = 0. 

As o"(-B) C o r (G') if and only if B C G, we can apply locality and derive 
V(B) = V(G) as needed. □ 

Lemmas 15 and 16 prove that (H,V) and (£>/~,<,7Y) are basis-equivalent, 
in the sense that B is a basis of G in (H, V) if and only if o~(B) is a basis of 
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Fig. 3. Hasse diagram from smallest enclosing circle of the vertices of a square. 

a(G) in (£>/ ~, <, H): Starting with a basis B of G in (if, V), Lemma 15 yields 
min(H cr(B)) = min(p| a(G)). This cr(B) is inclusion-minimal w.r.t. u{G), since 
otherwise Lemma 16 would yield a contradiction to the inclusion-minimality 
of B w.r.t. G (where we again use that cr(B) C a(G) if and only if B C G). 
The reasoning for inclusion-minimality in the other direction is analogous. 
This concludes the proof of Proposition 14. □ 

Propositions 13 and 14, together with the fact that every concrete LP-type 
problem can be transformed into an abstract one (as described below Defini- 
tion 4), yield Theorem 8. 



3.4 Examples 



Here we present some particular abstract LP-type problems and we demon- 
strate the construction (via acyclic violator spaces) of their concrete represen- 
tations. 

Let a, b, c and d be the vertices of a unit square (in the counterclockwise order); 
let H = {a,b,c,d}. For G C H let w(G) be the radius of the smallest circle 
enclosing all the points of G (for G = put w (G) = —00). The corresponding 
acyclic violator space is described by the following table: 



G 





a 


b 


c 


d 


ab 


ac 


ad 


V(G) 


abed 


bed 


acd 


abd 


abc 


cd 





be 


G 


be 


bd 


cd 


abc 


abd 


acd 


bed 


abed 


V(G) 


ad 





ab 


















The bases are 0, a, b, c, d, ab, ac, ad, be, bd, cd; the only equivalent pair is 
ac ~ bd. There is no inconvenience concerning differences between < on sets 
and equivalence classes and <i, the ordering <i is given by the Hasse diagram 
in Figure 3. 
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Fig. 4. Illustration example - linear programming. 

As the linear extension < of <i we may choose ®<a<b<c<d<ab< 
be < cd < ad < [ac\. Finally, the concrete representation S is as follows: 



h 


a 


b 


c 


d 


S(h) 


a, ab, ad, [ac] 


b, ab, be, [ac] 


c, be, cd, [ac] 


d, cd, ad, [ac] 



In the geometric view that we have mentioned earlier, S(a) corresponds to the 
set of all "canonical" (i.e., basic) balls that contain the point a (inside or on 
the boundary). The same holds for the other points. 



As the other example, consider the following LP problem in the positive or- 
thant (rotated by 45 degrees for convenience). Beside the restriction to the 
positive orthant, the constraints are the four halfplanes depicted in Figure 4. 
The optimization direction is given by the arrow. 

Here the violator space bases are 0, a, b, c, d, ac, ad, be, bd; the equivalence 
classes are O = 0, A = {a}, B = {b}, C = {c}, D = {d} and Q = {ac} ~ 
{ad} ~ {be} ~ {bd}. Note that the equivalence classes correspond to the 
points in the plane. We have O <i B <i A <i Q and O <\ C <i D <i Q; we 
choose < to be 0<B<A<C<D<Q. The concrete representation is 



h 


a 


b 


c 


d 


S(h) 


A,Q 


A,B,Q 


C,D,Q 


D,Q 



Here we may interpret S(a) as the set of all "canonical" points lying in the 
halfplane a. 

To see why we allow Ti in the definition of a concrete LP-type problem to 
be a multiset, consider the abstract LP-type problem with H = {a, b} and 
w(G) = for every G C H . The only basis is and it is not violated by 
any h G H. Thus we have S(a) = S(b) = {[0]}. If we do not allow TC to be 
a multiset, we have H = {S(a), S(b)} = {{[0]}} with only one constraint; it 
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seems improper to define this to be basis-equivalent to H. We could alter the 
construction of 7i and get S(a) = {0}, S(b) = {0, 1}, which does represent the 
original abstract LP-type problem with Ti being a set; however, we believe that 
our definition catches the structure in a more straightforward way, although 
it may seem unusual at first glance. 



4 Clarkson's Algorithms 

We show that Clarkson's randomized reduction scheme, originally developed 
for linear programs with many constraints and few variables, actually works 
for general (possibly cyclic) violator spaces. The two algorithms of Clarkson 
involved in the reduction have been analyzed for LP and LP-type problems 
before [8,9,10]; the analysis we give below is almost identical on the abstract 
level. Our new contribution is that the combinatorial properties underlying 
Clarkson's algorithms also hold for violator spaces. 

We start off by deriving these combinatorial properties; the analysis of Clark- 
son's reduction scheme is included for completeness. 

4-1 Violator spaces revisited 

We recall that an abstract LP-type problem is of the form (H, w, W, <). In this 
subsection we will view a violator space as an "LP-type problem without the 
order <", i.e., we will only care whether two subsets F and G, F C G C H , 
have the same value (and therefore the same violators, see Lemma 12), but not 
how they compare under the order <. It turns out that the order is irrelevant 
for Clarkson's algorithms. 

Even without an order, we can talk about monotonicity in violator spaces: 
Lemma 17 Any violator space (H, V) satisfies 

Monotonicity: V(F) = V(G) implies \J{E) = V(F) = V(G), 
for all sets F C E C G C H. 

PROOF. Assume M{E) ^ V(F), V(G). Then locality yields ^ E n V(F) = 
E fl V(G) which contradicts consistency. □ 

Recall Definition 7: A basis is a set B satisfying B fl V(F) ^ for all proper 
subsets F of B. A basis of G is an inclusion-minimal subset of G with the same 
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violators. This can be used to prove the following observation, well-known to 
hold for LP- type problems [9]. 

Observation 18 Let (if, V) be a violator space. For R C if and all h E if , 

we have 

(%) y(R) ^ V(R u {h}) if and only if h E V{R), and 
(ii) V(i?) 7^ V(i? \ {h}) if and only if h is contained in every basis of R. 

An element h such that (ii) holds is called extreme in R. 



PROOF, (i) If h i V(R), we get V(i?) = V(i? U {h}) by Lemma 11. If 
h E V(_R), then V(i?) ^ V(i?U {h}) is a consequence of consistency applied to 
G = RU{h}. (ii) ifV(i?) = V(i?\ {h}), there is a basis B of R \ {h}, and this 
basis is also a basis of R not containing h. Conversely, if there is some basis B 
of R not containing h, then V(i?) = V(R\{h}) follows from monotonicity. □ 

We are particularly interested in violator spaces with small bases. 

Definition 19 Let (H, V) be a violator space. The size of a largest basis is 
called the combinatorial dimension 5 = S(H,\/) of (if, V). 

Observation 18 implies that in a violator space of combinatorial dimension 
5, every set has at most 5 extreme elements. This in turn yields a bound for 
the expected number of violators of a random subset of constraints, using the 
sampling lemma [12]. 

Lemma 20 [12] Consider a triple (H,w,W), where w is a function mapping 
subsets of the set H to the set W (not necessarily ordered). For R C H , we 
define 

V{R) :={hEH\R: w(R) ^ w(R U {/*}), 
X(R) :={hER: w(R) ^ w(R \ {h}). 

ForO < r < \H\, letv r be the expected value of\V(R)\, for R chosen uniformly 
at random among all subsets of H with r elements. x r is defined similarly as 
the expected value of \X(R)\. Then for < r < n, the following equality holds. 

n — r r + 1 

To apply this in our situation, we fix a set W C H, and we define w(R) = 
\I(W\JR). Since then \X(R)\ < 5 for all R, the following Corollary is obtained. 



17 



Corollary 21 Let (H, V) be a violator space of combinatorial dimension 5 
and W C H some fixed set. Let v r be the expected number of violators of the 
set W U R, where R C H is a random subset of size r < n = \H\. Then 



4-2 The Trivial Algorithm 

Given a violator space (H, V) of combinatorial dimension S, the goal is to find 
a basis of H . For this, we assume availability of the following primitive. 

Primitive 22 Given G C H and h G H\G, decide whether h e V(G). 

Given this primitive, the problem can be solved in a brute-force manner by 
going through all sets of size < 5, testing each of them for being a basis of H . 
More generally, B C G is a basis of G if and only if 



Consequently, the number of times the primitive needs to be invoked in order 
to find a basis of H is at most 



The next two subsections show that this can be substantially improved. 

4-3 Clarkson's First Algorithm 

Fix a violator space (if, V) of combinatorial dimension 5, implicitly specified 
through Primitive 22. Clarkson's first algorithm calls Clarkson's second algo- 
rithm (Basis2) as a subroutine. Given G C H, both algorithms compute a 
basis B of G. 

Basisl(G): 

(* computes a basis B of G *) 
IF \G\ < 95 2 THEN 

RETURN Basis2(G) 
ELSE 



V r < 5 



n — r 



r + 1 



he\/{B\{h}), VheB, 
h<£V(B), VheG\B. 
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W := 
REPEAT 

choose R to be a random r-element subset of G, R e 

C : = Basis2(WU R) 

V := {heG\C-.he V(C)} 

IF |V| < 2tJ\G\ THEN 

W:=^UV 
END 
UNTIL V = 
RETURN C 
END 



Assuming Basis2 is correct, this algorithm is correct as well: if B is a basis 
of W U .R C G that in addition has no violators in G, B is a basis of G. 
Moreover, the algorithm augments the working set W at most 5 times, which 
is guaranteed by the following observation. 

Observation 23 If C C G and G n V(C) ^ 0, toen G n V(G) contains at 
least one element from every basis ofG. 

PROOF. Let B be a basis of G. Assuming 

= BnGnv(C) = Bnv(C), 



consistency yields C fl V(C) = 0, implying (_B U C) fl V(C) = 0. From locality 
and monotonicity (Lemma 17), we get 

V(C) =V(BUC) = V(G), 



meaning that G n V(G) = Gfl V(C) = 0, a contradiction. □ 

It is also clear that Basis2 is called only with sets of size at most Z8yJ\G\. 
Finally, the expected number of iterations through the REPEAT loop is bounded 
by 25: by Corollary 21 (applied to (G,V| G )) and the Markov inequality, the 
expected number of calls to Basis2 before we next augment W is bounded by 
2. 

Lemma 24 Algorithm Basisl computes a basis ofG with an expected number 
of at most 2S\G\ calls to Primitive 22, and an expected number of at most 26 
calls to Basis2 ; with sets of size at most 35J\G\. 
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4-4 Clarkson's Second Algorithm 

This algorithm calls the trivial algorithm as a subroutine. Instead of adding 
violated constraints to a working set, it gives them larger probability of being 
selected in further iterations. Technically, this is done by maintaining G as a 
multiset, where fi(h) denotes the multiplicity of h (we set fJ.(F) := J2heF M^))- 
Sampling from G is done as before, imagining that G contains fj,(h) copies of 
the element h. 

Basis2(G): 

(* computes a basis B of G *) 
IF |G| < 65 2 THEN 

RETURN Trivial(G) 
ELSE 

r := 65 2 

REPEAT 



C := Trivial^) 
V := {h G G\C:h e V(C)} 
IF fi(V) < n(G)/3S THEN 
/i(/i) := 2/x(/i), /i G V 
END 
UNTIL V = 
RETURN C 
END 

Invoking Corollary 21 again (which also applies to multisets as we use them), 
we see that the expected number of calls to Trivial before we next reweight 
elements (a successful iteration), is bounded by 2. It remains to bound the 
number of successful iterations. 

Lemma 25 Let k be a positive integer. After k5 successful iterations, we have 
2 k < n{B) < \G\e k/3 , 

for every basis B of G. In particular, k < 3 In |G|. 

PROOF. Every successful iteration multiplies the total weight of elements 
in G by at most (1 + 1/35), which gives the upper bound (not only for /i(-B) 
but actually for n(G)). For the lower bound, we use Observation 23 again to 
argue that each successful iteration doubles the weight of some element in 
B, meaning that after kS iterations, one element has been doubled at least k 



choose random R G 
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times. Because the lower bound exceeds the upper bound for k > 3 In the 
bound on k follows. □ 

Summarizing, we get the following lemma. 

Lemma 26 Algorithm Basis2 computes a basis ofG with an expected number 
of at most 6S\G\\n\G\ calls to Primitive 22, and expected number of at most 
65m |G| calls to Trivial, with sets of size 65 2 . 

4-5 Combining the Algorithms 

Theorem 27 Using a combination of the above two algorithms, a basis of H 
in a violator space (H, V) can be found calling Primitive 22 expected 

0(5n + 6°V) 
many times. 



PROOF. Using the above bound for the trivial algorithm, Basis2 can be 
implemented to require an expected number of at most 

0(5 log \G\(\G\ + 5° {5) )) 

calls to the primitive. Applying this as a subroutine in Basis l^) with \H\ = 
n, \G\ is bounded by and we get an overall expected complexity of 

O (8n + 5 2 (log n{6 + S° (5) ) )) 

in terms of the number of calls to Primitive 22. The terms 8 2 \ogn and 
<5 2 logn 5°^ are asymptotically dominated by either Sn or 5 0( - s \ and we get 
the simplified bound of O [Sn + 5°^). □ 



5 Grid USO as Models for Violator Spaces 

We show in this section that the problem of finding the sink in a 5-dimensional 
grid unique sink orientation [16] can be reduced to the problem of finding the 
(unique) basis of a violator space of combinatorial dimension 5. 
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Unique sink orientations of grids arise from various problems, including linear 
programming over products of simplices and generalized linear complementar- 
ity problems (GLCP) over P- matrices [16]. The GLCP has been introduced 
by Cottle and Dantzig [30] as a generalization of the well known LCP [31]. 
There are also applications in game theory; for instance [25,26,27] show how 
parity, mean-payoff, and simple stochastic games are related to grid USO. 



5.1 Grid USO 
Fix a partition 

n = (n!,...,ni) 

of the set H := {1, . . . ,n} into 5 nonempty subsets, where we refer to IT as 
the block i. A subset J C H is called a vertex if | J n ILJ = 1 for all i. The 
vertices naturally correspond to the Cartesian product of the IT. Let V be the 
set of all vertices. 

In the following definition, we introduce the grid spanned by subsets LT^ whose 
union is G C H. The vertex set of this grid contains all vertices J C G 
(J G V), with two vertices being adjacent whenever they differ in exactly two 
elements. 

Definition 28 The 5 -dimensional grid spanned by G C H is the undirected 
graph Q(G) = (V(G),£(G)), with 

V(G) :={JeV:JC G}, £{G) := {{J, J'} C V(G): \J®J'\ = 2}. 
Here, © is the symmetric difference of sets. 

V(G) is in one-to-one correspondence with the Cartesian product 

5 

HG h G 4 :=Gnn 8) 
i=i 

and the edges in £(G) connect vertices in V(G) whose corresponding tuples 
differ in exactly one coordinate. See Figure 5 left for an example of a grid. 

Note that Q{G) is the empty graph whenever Gi = G D Ilj = for some i. We 
say that such a G is not U-valid, and it is U-v alid otherwise. 
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Fig. 5. A 3-dimensional grid Q{H) with H = {1,...,7} where 
n = ({1, 2, 3}, {4, 5}, {6, 7}) and a USO of it. 

A subgrid of Q(G) is any graph of the form Q(G'), for G' C G. 

Definition 29 An orientation ip of the graph Q := Q{H) is called a unique 
sink orientation (USO) if all nonempty subgrids of Q have unique sinks w.r.t. 



We are interested in finding the sink in a USO of Q as fast as possible, since 
the sink corresponds to the solution of the underlying problem (the P-matrix 
GLCP, for example). Our measure of complexity will be the expected number 
of edge evaluations, see [16]. An edge evaluation returns the orientation of the 
considered edge and can typically be implemented to run in polynomial time 
(depending on the underlying problem). In the remainder of this paper, we 
derive the following theorem. 

Theorem 30 The sink of a unique sink grid orientation can be found by 
evaluating expected O (5n + 5°^) edges. 



Note that a USO ip can be cyclic (see the thick edges in Figure 5 right). If 

ip induces the directed edge (J, J'), we also write J — > J'. Any USO can be 
specified by associating each vertex J with its outgoing edges. Given J and 
j G H \ J, we define J > j to be the unique vertex J' C J U {j} that is 
different from J, and we call J' the neighbor of J in direction j. Note that J 
is a neighbor of J' in some direction different from j. 

Definition 31 Given an orientation ip of Q , the function : V — >• 2 H , de- 
fined by 



s^J) := {jEH\J:J^J>j}, 



(1) 



is called the outmap ofip. 



By this definition, any sink w.r.t. ip has empty outmap value. 
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5.2 Reduction to Violator Spaces 



Let us fix a unique sink orientation of Q. Given a Il-valid subset G C H, 
we define sink(G) G V(G) to be the unique sink vertex in Q(G). For a subset 
G that is not H-valid, let 

G:= (J n - 



Thus G is the set of elements occurring in blocks of II disjoint from G. 
Definition 32 For G CH, define 



V(G) 



s^(sink(G)), if G is Il-valid 

G, if G is not U-valid. 



Theorem 33 The pair (H, V) from Definition 32 is a violator space of com- 
binatorial dimension S. Moreover, for all U-valid G C H, the unique sink of 
the subgrid Q(G) corresponds to the unique basis of G in (H,V). 



PROOF. For every G C H, consistency holds by definition of sink(G), s^(J) 
and G. In order to prove locality for F C G C H, we look at three different 
cases. 



G is not Il-valid. Then, F C G is not IT-valid either. The condition = 
G fl V(F) = GflF means that F is disjoint from the same blocks as G. This 
implies G = F, hence V(G) = V(F). 



G and F are both II- valid. Then ^(F 1 ) is a nonempty subgrid of G(G), 
and G fl V(F) = means that the sink of Q(F) has no outgoing edges into 
G(G). Thus the unique sink of Q(F) is also a sink of Q(G) and therefore the 
unique one. This means that sink(G) = sink(F), from which V(GQ = V(F) 
follows. 



G is Il-valid, F is not FI-valid. Then the condition G fl V(F) = can 
never be satisfied since V(F) = F contains at least one full block IT, and 
G, = G n IT ^ 0. 
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Next we prove that a largest basis in (H, V) has at most 5 elements. For this, 
let G C H be a set of size larger than S. If G is Il-valid, we have 

V(G) := s^(sink(G)) = s v ,(sink(sink(G'))) =: V(sink(G)), 

since J = sink (J) for any vertex J. This means that G has a subset of size <5 
with the same violators, so G is not a basis. 

If G is not Il-valid, we consider some subset B that contains exactly one 
element from every block intersected by G. By definition, we have G = B and 
V(G) = V{B). Since B has less than 5 elements, G cannot be a basis in this 
case, either. 

It remains to prove that for G being Il-valid, the vertex sink(G ! ) is the unique 
basis of G in (H,V). We have already shown that V(G) = V(sink(G)) must 
hold in this case. Moreover, V(sink(G)) contains no full block Ilj. On the other 
hand, any proper subset F of sink(G) is not Il-valid, so its violator set does 
contain at least one full block. It follows that V(F) ^ V(sink(G)), so sink(G) 
is a basis of G. The argument is complete when we can prove that no other 
vertex J C G is a basis of G. Indeed, such a vertex J is not a sink in G{G), 
meaning that G n V(J) ^ 0. This implies V(J) ^ V(G). □ 

Note that the global sink of the grid USO corresponds to the unique ^-element 
(and Il-valid) set B with V(_B) = 0. This is exactly the set output by the call 
Basisl(if) of Clarkson's algorithms, when we apply it to the violator space 
constructed in Definition 32. 

Primitive 22 corresponds to one edge evaluation in the USO setting. With 
Theorem 27, we therefore have proved Theorem 30. For small 5, the running 
time given in the theorem is faster than the one from the Product Algorithm 
[16] which needs expected 0(5\n + H^) edge evaluations, where H n is the n-th 
harmonic number. 



6 Conclusions 

We introduced violator spaces as a new framework for optimization problems 
and showed that acyclic violator spaces are equivalent to abstract and con- 
crete LP-type problems. It turned out that the explicit ordering inherent to 
LP-type problems is not necessary in order to capture the structure of the un- 
derlying optimization problem. Violator spaces are more general than LP-type 
problems, yet Clarkson's algorithms still work on them. 

The Sharir-Welzl algorithm is also applicable for violator spaces in a straight- 
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forward way. However, the most obvious translation of this algorithm to the 
setting of violator spaces is not even guaranteed to finish, since for a gen- 
eral violator space it may run in a cycle and the subexponential analysis thus 
breaks down. 

We have seen that unique sink orientations are models for possibly cyclic viola- 
tor spaces, and with Clarkson's algorithms we therefore have a fast scheme to 
solve fixed dimensional USO problems like the generalized linear complemen- 
tarity problem with a P-matrix. The GLCP with a P-matrix has in general 
a cyclic structure and therefore gives rise to a cyclic USO. A violator space 
obtained from a cyclic USO is again cyclic. It is interesting that there are no 
cycles in a 2-dimensional grid USO [16]. Whether the same is true for violator 
spaces of combinatorial dimension 2 is an open question. 
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